Transformers may have sufficient supervision for generalization.

These limitations hinder GNNs' ability to capture long-range dependencies in graphs, ultimately

However, compared with the huge success of generative models in natural language processing [Tou-vron et al., 2023] and computer vision [Rombach et al., 2021], graph generation is faced with many

MPNNs, graph transformers do not rely on fixed graph structure information.

The formulation is very similar to the method for learning positional node embeddings. Asynthetic molecular graph regression dataset, where thepredictedscore isgivenby the subtraction of computationally estimated propertieslogP SA. Thetask is to classify the nodes into 2 communities, testing the GNNs ability to recognize predetermined subgraphs. For the training parameters, we employed an Adam optimizer with alearning rate decay strategy initializedin{10 3,10 4}asper[15],withsomeminormodifications: ZINC[15]. We selected aninitial learning rateof7 10 4 and increased thepatiencefrom 10 to 25 to ensure convergence.

The Transformer architecture has achieved remarkable success in a number of domains including natural language processing and computer vision.

Graphs arise across diverse domains, from biology and chemistry to social and information networks, as well as in transportation and logistics. Inference on graph-structured data requires methods that are permutation-invariant, scalable across varying sizes and sparsities, and capable of capturing complex long-range dependencies, making posterior estimation on graph parameters particularly challenging. Amortized Bayesian Inference (ABI) is a simulation-based framework that employs generative neural networks to enable fast, likelihood-free posterior inference. We adapt ABI to graph data to address these challenges to perform inference on node-, edge-, and graph-level parameters. Our approach couples permutation-invariant graph encoders with flexible neural posterior estimators in a two-module pipeline: a summary network maps attributed graphs to fixed-length representations, and an inference network approximates the posterior over parameters. In this setting, several neural architectures can serve as the summary network. In this work we evaluate multiple architectures and assess their performance on controlled synthetic settings and two real-world domains -- biology and logistics -- in terms of recovery and calibration.

While tokenized graph Transformers have demonstrated strong performance in node classification tasks, their reliance on a limited subset of nodes with high similarity scores for constructing token sequences overlooks valuable information from other nodes, hindering their ability to fully harness graph information for learning optimal node representations. To address this limitation, we propose a novel graph Transformer called GCFormer. Unlike previous approaches, GCFormer develops a hybrid token generator to create two types of token sequences, positive and negative, to capture diverse graph information. And a tailored Transformer-based backbone is adopted to learn meaningful node representations from these generated token sequences. Additionally, GCFormer introduces contrastive learning to extract valuable information from both positive and negative token sequences, enhancing the quality of learned node representations. Extensive experimental results across various datasets, including homophily and heterophily graphs, demonstrate the superiority of GCFormer in node classification, when compared to representative graph neural networks (GNNs) and graph Transformers.